Fall Detection Using mmWave

Table of Contents

Introduction

The objective of this experiment is to evaluate the ability of TI mmWave Sensor to detect rapid height changes in tracked people in real environments and raise an alert upon sensing a fall event.

In many circumstances, it can be valuable to detect the stance of a person with mmWave sensors. For example, in an elderly care scenario, the mmWave sensor can detect the stance, as well as detect dangerous events like falls, without compromising the privacy of the individual. The sensor delivers 3D position, velocity, and acceleration of tracked opbjects/people and while the target data gives information about the location and size of the person, it is not precise enough to be used to determine the identity or take an image of the person. The goal of this experiment is to demonstrate how this information can be used to detect falls in real household environments where falls are likely to occur like a living room or a bathroom. We will show how a simple delta position value monitoring for rapid position changes can effectively detect human falls, discuss algorithm changes for different use cases, and discuss the range limits on accurately detecting falls.

Setup

The following software and hardware setup was used to complete the experiment:

Hardware Setup

• This experiment was conducted inside various rooms, ranging from smalltiled bathroom to a larger living room.
• An IWR6843ISK-ODS was mounted to an Industrial Carrier Board (MMWAVEICBOOST). This setup will be referred to as ODS.
• An IWR6843ISK was mounted to an Industrial Carrier Board (MMWAVEICBOOST). This setup will be referred to as ISK.
  • The ODS antenna has better angular accuracy along the Elevation plane than the ISK antenna. This will allow the ODS to more accurately determine small changes in height in close range applications.
  • The ISK antenna performs better at longer ranges, and was used to test the maximum range of fall detection for large rooms/outdoor applications.
• The ODS and ISK were mounted on top of a tripod or the back of a door, depending on what the room size allowed, at a height of 2m.
• The ODS and ISK were mounted with 15 degree downtilt, so that the antenna could accurately view the entire room.
• A camera was placed behind the sensor to capture a base truth.

Software Setup

🛑 Software Versions
This experiment’s software compatibility has been affected by updates made to the mmWave Radar Toolbox. In order to use this experiment as described, please use the binary and visualizer from the mmWave Indstrial Toolbox Version 4.11.0

• The ODS was flashed with the 3D_people_count_68xx_demo binary, which can be found in the prebuilt binaries folder of the 3D People Counting Demo.
• The mmWave Fall Detection GUI was used to show the tracking and height graph output. The install and source file for this are included in the Radar Toolbox download, and located in the People Counting Visualizer folder at the below path. ” \tools\visualizers\Industrial_Visualizer ”
• The GUI was rotated so that the GUI had the same view point as the camera, which was directly behind the EVM.
• The attached sensor configuration was used for the ODS board: ODS_6m_smallRoom.cfg
• The attached sensor configuration was used for the ISK board: ISK_6m_default.cfg
• These configurations are the same configurations optimized for 3D people counting, but the boundary boxes were modified to reflect the size of the room being measured. This is a very important step to minimize reflections and ghost tracks.
• Note: This experiment was tested with the 3D People Counting demo, as detailed above. While the results below are mostly using the ODS antenna, this experiment is compatible with all the boards supported in the 3D people counting lab, linked above. If using a different board, the corresponding configuration file should also be used.

Procedure

For each mounting height, the following steps were followed:

1. Fall detection demo is configured and begins running
2. One person enters the scene and moves in front of the sensor
3. The person walks around the scene to verify accurate tracking.
4. The person simulates fall events in various methods.
5. The fall events were repeated in various rooms and with typical objects like shower curtains obstructing camera view.

During the course of the experiment in each scene, the fall detection software was left running.

Observations

GUI output will look like the following:

Included in the video clips are camera footage to show the correlation between the sensor output and human movement. The GUI shows the movement of the tracked person through the cube drawn inside the boundary box in the plotted area. The right hand side of the GUI shows the measured height of the tracked person, along with an average and short term delta height. Average height is a rolling average over the past 10 frames. The height delta compares the average height of the current frame to the average height of 10 frames prior (0.5 seconds). Controlled movements like sitting or kneeling down do not show a fast enough change in height to trigger a fall event.

Scene 1: Living Room

A typical living room environment was tested with sitting and falling around a couch. The room is about 5m long, with the boundary box being set at the perimeter of the room.

When transitioning from standing to sitting, a change in height is recognized, but is not fast enough to trigger a fall event.

When a person falls from standing, the change in height is rapid enough to be seen as different than sitting and triggers a fall event.

Multiple height adjustments can still be detected and trigger fall events. Transitioning from sitting to standing shows an increase in height, but a following fall event is still detected.

Scene 2: Bathroom

Fall detection with mmWave radar in a bathroom environment is very helpful, as the mmWave sensor can be monitoring a bathroom without the same privacy concerns as a camera system. Fall events are also likely to occur in bathrooms due to slick floors or wet surfaces. Because of the relatively small size of many bathrooms, and reflective tile or vinyl flooring, mmWave sensors can be prone to picking up reflected signals and ghost images, which could lead to false tracking and fall events. But by matching the boundary box settings to the size of the bathroom, the tracking algorithm operating on the mmWave sensor can intelligently ignore these false signals and maintain accurate fall detection of people while in the small room.

In the video clip above, the tracking works very well in the small bathroom with tile floors. The tracker can distinguish between a sitting motion and a fall event on the floor.

As long as the person remains inside the boundary box, the tracker maintains location and height estimation even when the person is behind a thin barrier, such as a shower curtain. The above clip shows the tracker can still detect a fall event even when a traditional camera would not be able to see through the opaque barrier.

Scene 3: Outdoors

The outdoor environment was only used to demonstrate the range capabilities of the fall detection. The ISK module was used for this test, as it has better range capabilities. But the limited vertical resolution of the antenna pattern begins to become problematic at longer ranges, as the minimum change in estimated height comes close to the total height of the person.

Fall Detection Algorithm

The fall detection algorithm for this experiment was run in the python GUI, not on the sensor itself. It only requires the positional data of the tracked person and performs simple calculations to determine falls. If the average height falls by more than a threshold amount within 10 frames (0.5 seconds) a fall event is detected. The algorithm was left intentionally simple to be applicable to all environments and applications.

Conclusion

The IWR6843 3D tracking can be used to reliably track changes in absolute and relative height of a human. These height changes can be used to very quickly detect fall events, less than half a second. This can be used in hospital, patient care, household, or any other facility where fall events need to be monitored.